On Standardized Network Topologies For Network Research

Simulation has become the evaluation method of choice for many areas of computer networking research. When designing new or revised transport protocols, queuing methods, routing protocols, (just to name a few), a common approach is to create a simulation of a small to moderate scale topology and measure the performance of the new methodology as compared to existing methods. We demonstrate that simulation results using this approach can lead to very misleading, and even incorrect, results. The interaction between the large number of variables in these simulations can lead to results that vary widely from between different simulation topologies. This interaction can lead to conclusions that are valid only under a limited set of conditions and which are not valid under differing conditions. We give empirical evidence showing different conclusions when the same comparisons are done using differing topologies. We argue the need for a standardized taxonomy of simulation topologies that capture a significant and realistic range of values for the various variables that impact the performance of a simulated network. Networking researchers could then perform comparative analyses using the entire set of the standardized topologies, giving more consistent results, with higher confidence in the results.